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Abstract 

Humans and other animals base their decisions on noisy sensory input. Much work 
has therefore been devoted to understanding the computations that underly such deci¬ 
sions. The problem has been studied in a variety of tasks and with stimuli of differing 
complexity. However, the impact of correlations in sensory noise on perceptual judg¬ 
ments is not well understood. Here we examine how stimulus correlations together with 
correlations in sensory noise impact decision making. As an example, we consider the 
task of detecting the presence of a single or multiple targets amongst distractors. We 
assume that both the distractors and the observer’s measurements of the stimuli are 
correlated. The computations of an optimal observer in this task are nontrivial, yet 
can be analyzed and understood intuitively. We find that when distractors are strongly 
correlated, measurement correlations can have a strong impact on performance. When 
distractor correlations are weak, measurement correlations have little impact, unless the 
number of stimuli is large. Correlations in neural responses to structured stimuli can 
therefore strongly impact perceptual judgments. 


1 Introduction 


The perceptual system has evolved to extract ecologically meaningful information from sensory 
input. For example, in many mid- to high-level visual tasks the brain has to make categorical, 
global judgements based on multiple stimuli where the identity of any individual stimulus 
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is not of direct relevance. In a visual search task, the goal might be to detect whether a 
predefined target object is present in a scene that contains multiple objects. Complicating 
such tasks is the fact that noise corrupts sensory measurements, especially when observation 
time is short, or many objects are present. 

Much work has been devoted to modeling the decision processes by which the brain con¬ 
verts noisy sensory measurements of a set of stimuli into a judgement about a global world 
state, such as the presence or absence of a target. These models often focus on various decision 
rules that can be applied to the measurements. By contrast, the measurements themselves 
are usually modeled in a rather stereotypical fashion, namely as independent and normally 


distributed, (e.g. 

Peterson et al. 

(1954); 

Nolte and Jaarsma 

(1967); 

Pelli 

(1985 

); c 

draharn 

et al. 

(1987) 

Palmer et al. 

(1993) 

Baldassi and Burr 

(2000); 

Baldassi and Verghese 

(2002); 

van den Berg et al. 

(2012) 

Ma et al. 

(2011 

); 

Mazyar et al. 

(2012 

)). Both the assumption 


of independence and the assumption of Gaussianity can be questioned. Specifically, neural 


correlations can extend to distances as long as 4mm in monkey cortex (Ecker et al. 2010 


Cohen and Kohn, 

2011 

). This suggests that sensory measurements can be strongly corre- 

lated ( 

Rosenbaum et al. 

2010 

Chen et al. 

2006 

). Here we focus on the effects of violation of 


the assumption of independent measurements on performance in categorical, global perceptual 
judgements. 

To make such perceptual judgments, an observer needs to take into account the statistical 
structure of the stimuli and the structure of measurements. Consider a search task where 
a subject is required to detect a target among distractors. The effects of measurement cor¬ 
relations and stimulus correlations will be intertwined: If the distractors are identical on a 
given trial, then strong correlations between the measurements will help preserve their per¬ 
ceived similarity. Namely, an observer can group the distractor measurements and identify 
the target as corresponding to the outlying measurement. By contrast, when distractors are 
unstructured (independently drawn across locations), strong measurement correlations may 
have no effect on performance. Thus, measurement and stimulus correlations should not be 
considered in isolation. 

Here we examine how measurement and stimulus correlations impact the strategy and 
the performance of an ideal observer in a target detection task. We assume that on half the 
trials, one or more target stimuli are presented along with a number of distractors, whereas 
on the other half of trials, only distractors are presented. The task is to infer whether tar¬ 
gets are present or not. Importantly, we assume that the distractor stimuli are not drawn 
independently - for instance, in the extreme case the targets could be identical. Our ideal 
observer infers target presence based on measurements of the stimuli. We assume that these 
measurements are corrupted by correlated noise. In an extreme case, this noise is perfectly 
correlated and all measurements are perturbed by the same, random value. 

We provide an analytical study of the optimal decision rule to show that the interplay 
of measurement and stimulus correlations can be intricate. In general, if the stimuli are 
strongly correlated, then measurement correlations can strongly affect the performance of an 
ideal observer. When stimuli are weakly correlated, measurement correlations have a smaller 
impact. 

We expect that these insights hold more generally: Natural stimuli are structured, and their 
distributions are concentrated along low dimensional structures in stimulus space (GeEler, 
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2008). Correlations in measurement noise could thus help in the inferring parameters of 


interest. Ultimately, we thus expect our results to be relevant to modeling perceptual decision¬ 
making in natural scenes. 


2 Model description 

To examine how decisions of an ideal observer are determined by the statistical structure 
of measurements and stimuli we consider the following task. An observer is asked whether 
target stimuli are present among a set of distractor stimuli. Each stimulus, i, is characterized 
by a scalar, s* G M. The set of N stimuli presented on a single trial is characterized by 
the vector s = (si,S 2 ,-" , sat). For instance, stimuli could be pure tones characterized by 
their frequency, gratings characterized by their orientation, or ellipses characterized by the 
orientation of their major axis. A target is a stimulus with a particular characteristic, St- A 
target could be a vertical grating, or a pure tone at 440Hz. For simplicity, we assume that, if 
i is a target, then Si = St = 0, and stimulus characteristics are measured relative to that of a 
target. Stimuli that are not targets are distractors. For such stimuli Sj ^ 0 with probability 1. 
We will consider situations with single and multiple targets. 


p(T) p (s I T ) />(* I s) 



Figure 1: A schematic representation of Bayesian inference. Information about a parameter 
of interest, T, is encoded in a stimulus vector, s. The dimension of s could be much higher 
than the dimension of T. The observer makes a sensory measurement, x, of the stimulus, s, 
and must infer T from this measurement. This inference requires marginalization over s. 


The following treatment parallels the one we used earlier for independent measurements (Muz- 


yar et al. 2012, 2013 Bhardwaj et al. 2015). We denote target presence by T = 1, and absence 


by T = 0. We assume that targets are present with probability 0.5. When T = 0 (no targets) 
stimuli are drawn from a multivariate normal distribution with mean Ojv = (0, 0, • • • ,0), and 
covariance matrix, E s . The subscript denotes vector length, so On has N components. We 
can therefore write 

p(s|T) = J\f (s; Oat, E s ), (2.1) 

where A/"(s; /x, E) denotes the density of the normal distribution with mean /i and covariance 
E. For simplicity, we assume E s has constant diagonal and off-diagonal terms, so that 


S s = u 


1 Ps 
Ps 1 


Ps 

Ps 


( 2 , 2 ) 


Ps Ps ' 1 

The correlation coefficient, p s , determines the relation between the components of the stimulus. 
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If T = 1, then n > 1 targets are present. In this case we assign, with uniform probability, 
the target characteristic to n out of the total of N stimuli. This subset n targets is denoted 

by L. We denote by C the collection of all I 1 possible choices of the sets L of targets. 

The remaining N — n distractors are drawn from a multivariate normal distribution 
with mean Oa r_ n , and covariance matrix E s \l of dimension (N — n ) x (N — n ). Let = 
( s n, s t 2 ,''' > k ^ L denote the target stimuli, and s\ L = (s jl ,s h , ■ - - , s jN _ n ), j t L the 
distractors. We can therefore write 

p{s L \T = 1) = y^<S(gi), and p{s\l\T = 1) = J\f{s\ L ] 0 N -n, £ 8 \l). 

i£L 

Since the density p(s|T = 1 , L) is singular, we introduce the auxiliary covariance, 


{ (51s\Z,)iJ, if k j ^ L, 

V, if i = j e L, (2.3) 

0, if i £ L, or j G L, and i ^ j , 

where rj > 0. Therefore, 

p(s\T = 1, L) = lim Af(s- On, 

* 74-0 

We further assume that an observer makes a noisy measurement, Xj, of each stimu¬ 
lus, s t . This measurement can be thought of as the estimate of stimulus i obtained from 
the activity of a population of neurons that responded to the stimulus. We denote by 
x = (xi,X 2 , ••• ,Xn) the vector of N measurements. It is commonly assumed that these 
measurements are unbiased, and corrupted by additive, independent, normally distributed 
noise, so that p(x|s) = \\^ =1 Mixp, Si, cr^). Here we consider the more general situation where 
the measurements are unbiased, but noise could be correlated so that 


p(x|s) = A/"(x; s, 'Ey 


(2.4) 


We consider the particular case when X x has constant diagonal terms, a^., and off-diagonal 
terms, p x &l, so that 

1 Px ' ' ' Px 




Px 


1 


Px Px 


Px 

1 


(2.5) 


Note that 


P(x|T=l) = /p(x|s)j>(s|T=l)<is, 


and similarly for p(x|T = 0). We must thus marginalize over s to obtain these distributions. 
A simple computation (See Appendix 0 shows that p(x.\L,T = 1) = A/"(x; O^v, E° L + S x ), 
and p(x|T = 0) = AT(x; 0^, S s + S x ). 
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Figure 2: Stimulus and measurement distributions in the single target detection task with 
N = 2 stimuli and a s = 15 ,a x = 4. (A) (Top) Stimulus distributions on target present 
(left) and target absent (right) trials for p s = 0.5. On target present trials, the distribution 
is constrained to the axes, since one of the stimuli is a target. On target absent trials, the 
two stimuli follow a bivariate normal distribution. (Bottom) Distributions of measurements 
in the absence of measurement correlations, p x = 0. On both the target present (left), and 
target absent case (right), the measurement distribution inherits its shape from the stimulus 
distribution. (B) Overlap of measurement distributions for strongly correlated distractors, 
p s = 0.99. Measurement distributions on target present (black) and absent (gray) trials are 
shown for p x — 0 (top) and p x = 0.95 (bottom). The overlap between the measurement 
distributions decreases with increasing measurement correlation, p x . The decision boundary 
(solid line) therefore better separates the two distribution when p s and p x are both large. We 
used high correlations to bring out the difference between the distributions. Axes represent 
the total standard deviation of the measurements, a = \Jc d? + <j x . 
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3 Results 


Our goal is to describe how correlations between stimuli along with correlations between 
their measurements affect the decisions of an optimal observer in a target detection task. 
We examine how performance changes as both correlations between distractors and between 
measurements are varied. 

The stimuli, s, follow different distributions depending on whether T = 0 or T = 1. Mea¬ 
surement noise increases the overlap between the corresponding measurement distributions, 
p(x|T = 0) and p(x|T = 1). The higher the overlap between these two distributions, the more 
difficult it is to tell whether a target is present or not. However, correlations in measurement 
noise can reduce such overlap (See Fig. |2]B), even when noise intensity is unchanged. There¬ 
fore, the estimate of a parameter from a neural response depends not only on the level, a x , 


but also on the structure of measurement noise (Abbott and Dayan, 1999 Sompolinsky et al 


2001 Averbeck et al. 2006; Josic et al. 2009). 


An ideal observer makes a decision based on the sign of the log posterior ratio, 

p{T = l|x) p(x|T = 1) p(T = 1) 
d TD (x) = log ^^ = i og 9_2_-W + log FK 


' p(T = 0|x) 


1 p(x|T = 0) p(T = 0)’ 


(3.1) 


If cItd{x.) > 0, the observer infers that a target is present. Note that 

p(x|T = 1) = J^p(x|L,T = 1 )p(L). 

L£C 


Given that the prior probability that L is a set of targets is uniform gives p(L) = 
and 


N 


n 


d T d(x) = log 


P(x \T = 1) 

p(x|T = 0) 


= log 


-l 


E 

LeC 


p(x|L,r = i) 
p(x|T = 0) 


(3.2) 


The decision variable thus depends on the sum of the normalized probabilities that a mea¬ 
surement in x is made, given that the target set is L. 

Note also that 

p(L|x ,T = 1) ocp(x|L,T = 1). 

Therefore, the decision variable can also be interpreted as a sum of likelihoods that L is a 
target set, given a measurement x. Thus the decision is directly related to the posterior 


distribution over the target sets L: The summands in Eq. (3.2) correspond to the evidence 
that L is a set of targets. 

The distribution of measurements, x, depends on whether a target or targets are present. 
The conditional distributions of measurements, p(x|T = 0) and p(x|L,T = 1), are Gaussian 
with all means equal to 0, and covariances C = £ s + £ x and Cl = + £ x , respectively. 

We therefore have (see Appendix [A] for details) 


dr d(x) = log 



exp 


Lee 


'^x T (C^ 1 — C” 1 ) x 


(3.3) 
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This decision variable depends on the model parameters, and the measurement, x. The 
total number of stimuli, N, number of targets, n, and the variability, a 2 . The correlation, p s , 
between the distractors determine the structure of the stimulus, while the variability, a 2 , and 
correlation, p x , describe the distribution of sensory measurements. An ideal observer knows 
all these parameters. 


Setting the right hand side of Eq. (3.3) to 0 defines a nonlinear decision boundary in the 


space of measurements. This boundary separates measurements that lead to a “target present” 
decision (g?td(x) > 0) from those that lead to a “target absent” decision (g?td(x) < 0). For 
example, with a single stimulus, N — n — 1, the observer needs to decide whether or not a 
single measurement corresponds to a target. The decision variable has the form 


d T o(x) = log 


1 


X\ 


at 


2 °x 1 + ’ 


If the measurement, aq, differs sufficiently from 0, then c?td(x) < 0 . A measurement close to 
0 , gives g?td (x) > 0 . 

With more stimuli, the observer needs to take into account the known correlations between 
the stimuli and between the measurements. The decision variable depends in a complicated 
way on the parameters that describe the structure of the stimulus and the response. The 
explicit form of Eq. ( |3.3[ ), can be derived under the assumption of equal variances and equal 
covariances (See Appendix |A|) 

/ r / 

i 

M 


d TD (x) = log 


1 + (p s a 2 + p x a 2 )Nv fv L \ n 


7 L 


E 

Lee 


ex P —2 


V 


<j 2 s { 1- p s )v L vy x 2 


i£L 


I 


\ 


+ Pv 2 


Px°l 


i v l9l \ 

7i J • Ar - T 


XiXj + 2 ( /3v 


i,j£L 


p x a 2 x v L v 
7 L 


x^j + ( f3v~ 

ieL,j^L 


V 2 h 

1L 


E 

hj 


X iX j 


II 


III 


IV 


(3,4) 


The variables v and vj j represent scaled inverse variances corresponding to distractor and 


target stimuli. The parameters P,gL,h, and 7 l are given in Eqs. (A.4) and (A.5), and are 
defined in terms of a 2 , p s , a 2 ., and p x . Eq. (3.4) has a form that can be interpreted intuitively: 


1. Term 1 contains a sum of squares of individual measurements, x 2 , over the putative set 
of targets, L. The smaller this sum, the more likely that L contains targets. 


2. Term II contains the sample covariance about the known target value, St = 0, that is, 
^2ij£L( x i ~ s T)(xj ~ St) = EijeL x i x r absence of measurement correlations, 

p x = 0 , covariability about the target value has vanishing expectation for target mea¬ 
surements. Therefore the larger this sum, the less likely that the measurements come 
from a set of target stimuli. In the presence of measurement correlations, p x > 0, co¬ 
variability between target measurements is expected. Hence the prefactor in term II 
decreases with p x . 
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3. Term III is similar to term II, with the sum representing the sample covariance about the 
target value between putative target and non-target stimuli. The larger this covariance, 
the less likely that L or its complement contain targets. 

4. Term IV contains the sample covariance about the mean of measurements outside of 
the putative target set. If there are no targets, then all terms in the sum are expected 
to be large, regardless of the choice of L. However, if there are targets, then whenever 
the complement of L contains targets, some of the terms in the sum have expectation 
0. Hence the term again makes a smaller contribution if targets are present. 


While this provides an intuitive interpretation of the sums in Eq. (3.4), the expression is 
complex and it is difficult to precisely understand how an ideal observer uses knowledge of the 
generative model and the stimulus measurements to make a decision. We therefore examine 
a number of cases where Eq. (3.4) is tractable, and all the terms can be interpreted precisely. 
We also numerically examine performance in a wider range of examples. 


3.1 Single target, n = 1 


We start with the case when a single target is present at one of N locations. This case was con¬ 


sidered previously in the absence of correlations between the sensory measurements (Bhardwaj 


et al. 2015). 


We observe in Figs. [3]A. and [4j\ that the performance of an ideal observer is nearly inde¬ 
pendent of p s and p x when external structure is weak (p s < 1). Performance depends strongly 
on p x when distractors are strongly correlated, p s ~ 1. An ideal observer performs perfectly 
when p s = p x = 1 (Fig. [3p). 

Increased performance with increasing stimulus correlations, p s , accords with intuition 
that similar distractors make it easier to detect a target. However, correlations in measure¬ 
ment noise can play an equally important role and significantly improve performance when 
distractors are identical (See Fig. |3]B). 

Perfect performance when p s — p x — 1, can be understood intuitively. In this case mea¬ 
surements, Xi, of the stimuli are obtained by adding the same realization of a random variable, 
i.e. identical measurement noise to each stimulus value, Sj. In target absent trials, all mea¬ 
surements are hence identical. If the target is present, measurements contain a single outlier. 
An ideal observer can thus distinguish the two cases perfectly. 

We examine in more detail the cases of weak measurement noise, and then the case of 
comparable measurement noise and distractor variability. 


Weak measurement noise, <r‘ 

~ a 


< with highly correlated stimuli, p s 


1. When 

measurement noise is weak, discriminability between the “target absent” and “target present” 
conditions is governed by external variability, i.e. trial-to-trial variability of the stimuli. As 
noted, measurement correlations improve discriminability in the presence of strong stimulus 
correlations (see Figs. |3]4,C). When p s ~ 1, Eq. (3.4) can be approximated as (details in 









Appendix |B.1[): 


( 


d T d(x) «log 


1 N(l-p x ) 


N V N - 1 


V P(N,p x ) 


N 


exp 


i= 1 


2A^a2(! - p s ) 


(1 - Np x )x1 + 2.x-, ^ x-j - - f J2 x i 

j'A* VjA 



V 

ai(x.,N,p x ,cr x ) 


/ 

(3.5) 

The ideal observer hence uses knowledge about the measurement correlations, p x , in a decision. 

We first confirm the intuitive observation that an ideal observer performs perfectly when 
measurement noise is highly correlated, 1 — p x <C 1. In this case the exponential term in 


Eq. (3.5) is, to leading order in 1/(1 — p x ), 

cq(x, N, p x , a x ) « exp 


N - 1 


(Xi - Xi y 


(3.6) 


[2Nal(l-p x ) 

where x% = l/(N — l) x j i s sample mean of the measurements excluding the putative 
target, i. Hence, to make a decision, the ideal observer subtracts the mean of the N — 1 


measurements of putative distractors from that of the putative target. In Appedix B.l 


we 


show that on “target absent” trials, g(td( x ) A > — oo, as p x —> 1, and on target present trials, 
c(td( x ) — > oo, as p x —>■ 1. Hence performance improves as measurement correlations increase. 
This can also be seen in Fig. [2jB, as the overlap between the distributions p( x |T = 0) and 
p(x|T = 1) decreases with an increase in p x . 

When measurement correlations are absent, p x = 0, an ideal observer performs perfectly 


only when measurement noise is vanishing. The exponential in Eq. (3.5) now equals 


ch( x , N, p x , cr x ) = exp 


= exp 



(3.7) 


where x is the sample mean of all observations. The ideal observer therefore compares the 
sample mean over all measurements , x, with the sample mean over all observations excluding 
the putative target, x*. Since measurement noise is uncorrelated, averaging over all measure¬ 
ments is beneficial. This is a very different strategy than when measurement noise is highly 


correlated (Eq. (3.6)), and the ideal observer compares a single putative target measurement, 


Xj, to the sample mean of the remaining target measurements. As shown in Appendix B.l 
at intermediate values of measurement correlations, 0 < p x < 1, the ideal observer uses a 
mixture of these two strategies. 

Intuitively, an increased number of distractors make it more difficult to detect the target. 
An analysis of the decision variable in the limit N ^ oo shows that this is indeed the case 
(See Appendix |B.3[ ). 
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Measurement Correlation, p x Measurement, X j 

Figure 3: Performance of an optimal observer on a single target detection task with weak 
measurement noise, a 2 <C a 2 . (A) Proportion of correct responses as a function of stimulus 
correlation, p s and measurement correlation, p x , for N = 4 stimuli. (B) Proportion of correct 
responses as a function of measurement correlation, p x , in the case of intermediate stimulus 
correlation, p s = 0.5 (left) and strong correlation, p s = 1 (right) for N = 4 stimuli. (C) 
Decision boundary, d TD (x) = 0 (black solid line) and measurement distributions on target 
present (left) and target absent (right) trials for N = 2 and p s = 0.5. Here and henceforth 
dark gray dots correspond to measurements that result in a correct inference, and light gray 
dots correspond to measurements leading to an incorrect inference. The stimulus will result in 
a measurement that lies within the black dashed lines with probability 0.95. Other parameters 
used: a s = 15 and a x = 4. 
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Weak measurement noise, non-identical distractors, p s < 1. Measurement correla¬ 
tions have little effect on performance when stimulus correlations are weaker (See Fig. [3]4). 
Consider again p x 1, so that measurements are corrupted by adding a random, but nearly 
identical perturbation to the stimuli. An ideal observer uses the knowledge that all measure¬ 
ments are obtained by adding an approximately equal value to the stimulus. However, when 
stimulus correlations are weak, the target measurement is no longer an outlier. Correlations 
in measurement noise provide little help in this situation. 

These observations are reflected in the structure of the decision boundaries (g?td(x) = 0), 
and the distributions of the measurements (See Fig. [3^3). In the target absent (left column) 
and present trials (right column) the distributions of measurements, p(x|T), is shaped pre¬ 
dominantly by variability in the stimulus. Measurement correlations have little effect on this 
shape, and the decision boundary therefore changes little with an increase in p x . In con¬ 
trast when p s ~ 1, measurement correlations significantly impact the overlap between the 
distributions p(x|T = 1) and p(x|T = 0) as shown in Fig. [2j3. 

We can confirm this intuition about the role of measurement correlations by approximating 
the decision criterion when a 2 <C a 2 and p s < 1 (see details in Appendix |B.4[), 


d T d(x) « log 


/ ^(l ~ Ps)(l + (N ~ l)Ps) \ ^ 
jvy <7^(1 + (jv — 2 ) P< ) A, p 



(3.8) 


The strength of noise correlations, p x , does not affect the decisions of an ideal observer at 
highest order in (a 2 /a 2 ). This explains the approximate independence of performance on p x 
observed in Fig. [3j3 . In this case an ideal observer considers stimuli at each location separately, 
and weighs each measurement individually by its precision, 1 /a 2 (Green and Swets, 1966; 


Palmer, 1999|. 


When the number of distractors becomes larger, Eq. (3.8) is no longer valid. The observer 
compares measurements of all stimuli to make a decision. We return to this point below. 


Strong measurement noise, rr 2 — a 2 Increasing measurement noise trivially degrades 
performance. However, in the limit of perfect stimulus and measurement correlations, an ideal 
observer still performs perfectly for the reasons described earlier. 

Measurement correlations affect performance differently than in the case of weak measure¬ 
ment noise (See Fig. Eh ,C). Even with uncorrelated stimuli, p s = 0, performance increases 
slightly (approximately 5-6%) with p x . Surprisingly, for intermediate values of stimulus cor¬ 
relations, e.g. p s = 0.5, measurement correlations negatively impact performance. If mea¬ 
surement correlations are fixed at a high value, then the worst performance is observed at 
an intermediate value, 0 < p s < 1. The reason for this unexpected behavior is unclear, as 
Eq. (3.4) is difficult to analyze in this case. 

Generally, when measurement noise is strong, measurement correlations will change the 
shape of the measurement distributions p(x|T = 0) and p(x|T = 1), and hence impact deci¬ 
sions and performance. Note that when measurement correlations increase, the region cor¬ 
responding to g?td( x ) > 0 is elongated along the diagonal to capture more of the mass of 
the distribution p(x|T = 1) (See Fig. |4p). However, when measurement noise is high, the 
interactions between measurement and stimulus correlations are intricate. 
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Figure 4: Performance of an optimal observer on a single target detection task with strong 
measurement noise, a 2 = a 2 . (A) Proportion of correct responses as a function of p s and 
p x for N = 4 stimuli. (B) Performance as a function of measurement correlation, p x , when 
p s = 0 (left) and p s = 0.5 (right) for N = 4 stimuli. (C) Decision boundary (black solid line) 
and distribution of measurements, x on target present (left) and target absent (right) trials 
for N = 2 and p s = 0.5. Other parameters used: a s = <r x = 15. 
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3.2 Multiple targets, n > 1 

In the present task when multiple targets are present, they are all identical and hence perfectly 
correlated. Thus, regardless of the value of p s , on half the trials the stimuli will be strongly 
structured, and the density p(s|T = 1) concentrated on a low dimensional subspace. As a 
consequence, measurement correlations always impact performance. 

Regardless of stimulus correlations, an ideal observer performs perfectly when p x = 1 (See 
Fig. [5j4). Even when p s < 1, performance increases with p x (see Fig. |5]A). When p x = 1, all 
target measurements are identical. Hence, an ideal observer performs perfectly by checking 
whether n of the measurements, Xi , are equal. We only analyze the case p s < 1, since the case 
of perfectly correlated distractors is similar. 



Measurement Correlation, 


Measurement, at 1 


Figure 5: Performance of an optimal observer in a multiple target search task with weak 
measurement noise, a 2 -C cr 2 . (A) Proportion of correct responses as a function of stimulus 
correlation, p s , and measurement correlation, p x , with N = 4 stimuli and n — 3 targets. (B) 
Proportion of correct responses as a function of p x for p s = 0 (left) and p s = 0.5 (right) when 
N = 4 and n = 3. (C) Measurement distributions and decision boundary on target present 
(left) and target absent (right) trials for p s = 0.5 and N = n = 2. 


With weak measurement noise, Eq. (3.4) can be approximated as (see Appendix |B.4[ ): 
cZtd(x) «: 



/ (1 — Ar)(l + (N — l)p s ) | 

( ^ 2 (! ~ Ps)\ 

wv 

( 1 - (.V - n - l)p s )(l + (n- 1 )p s ) 

{a 2 x (l- p x )J 




n 


x > exp 

L&C 


2 (j2( 1 - Px ) l n 




np x 


i&L 


(1 + (n - 1 )p x ) \ n z 



(3.9) 
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An ideal observer takes into account measurement correlations for all values of p s even when 
measurement noise is low. Stimulus correlations only appear in the prefactor, and are not 
used in the comparison of the measurements inside the exponential. 

Interestingly, decisions in this case are based only on measurements of stimuli within the set 
of putative targets, L. In the absence of measurement correlations, a decision is based solely 
on the sample second moment of the n stimulus measurements about the target characteristic, 
st = 0, i.e. j./ ^ ieL 
moment indicates that L contains targets. 


1 / n x'f (the underlined term in Eq. (|3.9[)). A low value of this sample 


3 ieZ/’ 


In the limit of p x —» 1, the underlined term in Eq. (3.9) approaches the sample variance, 
i.e. the sample second moment about the sample mean, 


1 


n 




np x 


i&L 


1 + p x {n - 1) 



= s. 


ieL- 


(3.10) 


If measurement correlations are strong, the measurements of a set of target stimuli will be 
approximately equal (regardless of p s ). The ideal observer makes use of this knowledge by 
comparing the value of putative targets in the set L. If the sample variance of the measure¬ 
ments {xi}i e L is small, then it is likely that L is a set of targets. When p x = 1, an ideal 


observer performs perfectly (See Appendix B.5 for details). 

When 0 < p x < 1, the underlined term in Eq. (3.9) shows that the ideal observer takes 


an intermediate strategy by computing a second moment about a point between the target 
characteristic, St, and the sample mean. Interestingly, the larger the number of targets, the 
larger the weight on the sample mean, since the prefactor np x /( 1 + (n — 1 )p x ) increases with 
n for fixed p x . 

These observations are reflected in the distributions shown in Fig.[5p. The distribution of 
measurements, p(x|T = 1), move closer to the diagonal (aq = Xi ) as p x — > 1, and the overlap 
with the distribution of measurements, p(x|T = 0) decreases. In higher dimensions, for N 
stimuli and n targets, the measurement distributions, p(x|T = 1), is concentrated on the 
union of ( N — n + l)-dimensional subspaces when p x = 1: The target measurements lie on a 
line, while the N — n distractor measurements are distributed along the remaining directions. 

To conclude, when there are multiple targets part of the stimulus set is always perfectly 
correlated. When measurement correlations are high, the observer checks whether the mea¬ 
surements are similar to each other to make a decision. When measurement correlations are 
low, the observer compares the measurements to the known target value. Measurement corre¬ 
lations can again decrease the overlap between the conditional distributions of measurements, 
and significantly impact decisions and performance. For finite N, decisions are based on the 
comparison of measurements within a putative set of stimuli, L. We show next that when N 
is large, this is no longer the case. 


3.3 Larger number of targets and stimuli 

If we assume a fixed proportion, K , of the stimuli consists of targets, so that n = KN, 


then Eq. (3.4) simplifies considerably in the limit of large N. If we let c x = cr X (1 — p x ) and 
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c s = cr?(l — p s ), the exponential in Eq. (3.4) has the form, (See Appendix |B.6[) 


«l(x, N, c x , c s ) Rj exp 


1 

2 


V 


NK 


,, + 



(3.11) 


jy 


s ieL) and s2 are the sample variances of measurements form the putative target 


where 

set, L, outside of the putative target set, and over all N measurements, respectively. 

The different terms in this expression can be interpreted as earlier: Term 1 is the sample 
variance of measurements of the putative targets. If this variance is large, then the set L 
is unlikely to contain targets. Term 11 is the sample variance among all terms. If this term 
is large then all stimuli are dissimilar, and there is evidence that targets are present: For 
example, when distractors are correlated, and c s « 1, then the sample variance of all stimulus 
measurements is small only in the absence of targets. Finally, term III is the sample variance 
among putative distractors. If distractors are correlated, this term will be small if L contains 
targets, and hence stimuli outside L are distractors. The sign of the three terms agrees with 
this interpretation: Terms I and III are negative, and term II is positive. 

The main difference between the cases of large N, and the examples discussed previously 
is that an observer takes into account putative distractor measurements, i.e measurements 
outside the putative target set L. An exception is the case when measurement correlations 
are much stronger than distractor correlations, c x -C c s . In this case, the putative targets are 
more strongly structured, and hence only their measurements are used in a decision. When 
c s -C 1, or, equivalently, p s ps 1 , and distractors are strongly correlated, all three terms in 


Eq. (3.11) are comparable. In this case, ideal observers base their decision on the similarity, 


as measured by sample variance, of both putative distractor and target measurements. 

Importantly, the decision is made using distractor measurements, even when distractors 
are not perfectly correlated. Fig [6] shows that intermediate distractor correlations have an 
increasingly impact decisions with an increase in distractor number. Indeed, the higher the 
fraction of distractors, (1 — K ), the more weight is assigned to their sample variance (term 
III). This is unlike the case of small N, where distractor measurements are used only when 
they are perfectly correlated. 

4 Discussion 


We have shown that in a simple task the statistical structure of the stimulus as well as that of 
noise in perceptual measurements determine the strategy and performance of an ideal observer. 
Correlations in measurement noise can significantly impact performance, particularly when 
stimulus correlations are high: When the distribution of stimuli conditioned on the parameter 
of interest is concentrated in a small volume of stimulus space, the statistical structure of 
measurement noise can be of particular importance 


(Mazyar et al., 

2012, 

2013 

Bhardwaj 


et al. 2015). 


The impact of noise correlations on the inference of a parameter from neural responses 


has been studied in detail (Averbeck et al. 2006 Averbeck, 2009 Latham and Nirenberg 
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Figure 6: Normalized performance of an optimal observer with respect to p s = 0 as a function 
of stimulus correlation, p s , for different values of N in case of (A) p x = 0 and (B) p x = 0.5. 
Intermediate correlations between distractor measurements become increasingly important as 
the number of distractors increases from four to sixteen. Other parameters used: a s = 15, 
g x = 4, and n = 2. 


2005 Perkel et al. 1967; Schneidman et al., 2003 Sompolinsky et al. 2001). Frequently 
the parameter of interest was identified with the stimulus, and both were univariate. The 
estimation of the orientation of a bar in the receptive field of a population of neurons has 
been a canonical example. 

Reality is far more complex. Stimuli, such as a natural visual or auditory scene, are 
high dimensional and highly structured. Moreover, only some of the parameters are typically 
relevant. Intuitively, if noise perturbs measurements along relevant direction, i.e. along the 
directions of the parameters of interest, then estimates will be corrupted. Perturbations 


along irrelevant directions in parameter space have little effect (Moreno-Bote et al. 2014). 


Measurement correlations can channel noise into irrelevant directions, without decreasing 
overall noise magnitude, and thus improve parameter inference. 

This is difficult to study using general theoretical models without putting some constraints 
on the structure of measurement noise. We therefore considered a relatively simple, analyti¬ 
cally tractable example where both measurement and stimulus structure are characterized by 
a small number of parameters. We have used a similar setup to examine decision making in 


controlled search experiments (Mazyar et al. 2012, 2013 Bhardwaj et al., 2015). 


We assumed that measurement noise and measurement correlations can be varied indepen¬ 
dently. This is not realistic. For instance, it is known that changes in the mean, variability 


and covariability of neural responses can be tightly linked (Cohen and Kohn 2011 de la 


Rocha et al. 2007 Rosenbaum and Josic, 2011). It is thus likely that the statistics of mea¬ 


surement noise also change in concert. However, at present this relationship has not been well 
characterized. 

More importantly, noise from the periphery of the nervous system will limit the perfor¬ 
mance of any observer. It is therefore not possible that a simple change in measurement 
correlations can lead to perfect performance (Moreno-Bote et al., 2014). To address this ques¬ 


tion it would be necessary to provide a more accurate model of both the noise correlations in 


a recurrent network encoding information about the stimuli (Beck et al. 2011), as well as the 


resulting measurement correlations. This is beyond the scope of the present study. 
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We also made strong assumptions about the structure of measurement and stimulus cor¬ 
relations. We chose to restrict our analysis to positive correlations. The reason is that the 
requirement that a covariance matrix is positive definite implies restrictions on the range of 
allowable negative measurement and stimulus correlations (Horn and Johnson 2012). These 
restrictions depend on the number of stimuli, N, and complicate the analysis. To make the 
model tractable, we also assumed that all off-diagonal elements in the stimulus and measure¬ 
ment noise covariance matrices are identical. While we did not examine it here, heterogeneity 


in the correlation structure can strongly affect parameter inference (Shamir and Sompolinsky 
2006 Chelaru and Dragoi] 2008 Berens et al. 2011). 


(/) 

(D 




Figure 7: Performance comparison of an optimal observer as a function of measurement 
correlations in (A) a mean left/right discrimination task with N = 4 stimuli and (B) a target 
detection task with n — 3 targets. Other parameters used: a s = 15, a x = 4, and p s = 0.5. 


To end, we provide another illustration of the fact that measurement correlations can 
affect the performance of an ideal observer in different ways depending on the task: Suppose 
an observer is presented with N oriented stimuli, such as Gabor patches. The stimuli and 
measurements follow the same Gaussian distributions introduced earlier in this study. The 
observer is asked to perform one of the following two tasks: 1) Report whether the mean 
orientation of the stimuli is to the left or right of vertical; 2) Report whether a vertically 
oriented target is present or absent. In this case, a subset of the stimuli has vertical orientation 
on half the trials. 

The first task is a discrimination task, and the observer needs to to integrate informa¬ 
tion from different sources. When measurement correlations are high, it is more difficult to 


average out the noise between the stimulus measurements (Sompolinsky et ah, 2001; Zohary 


et al., 1994). The estimate of the average orientation is therefore degraded and performance 
decreases with an increase in measurement correlations (see Fig. [T]A. and Appendix [C]). The 
second is a detection task that requires extracting information that is buried in a sea of dis- 
tractors. As discussed above, in this case measurement correlations can increase performance 
if there is more than one target, or if the distractors are strongly correlated (see Fig. 03). 

In this example the stimuli and the measurements have the same statistical structure on 
target absent trials for the two tasks. However, the parameter of interest differs: In the 
first task the observer needs to estimate the average stimulus orientation, and in the second 
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determine whether a target is present. The distributions of measurements conditioned on these 
parameters are therefore also different, and are differently affected by measurement noise. 

The question of how measurement correlations impact decision making and performance 
does not have a simple answer (Hu et ah, 2014). Correlations in measurement noise can have a 
pronounced effect when the stimuli themselves are highly correlated, i. e. when they occupy a 
small volume in stimulus space. We have illustrated how in this case measurement correlations 
can help in separating the distribution of measurements conditioned on a parameter of interest. 
Similar considerations will be important whenever we try to understand how information can 
be extracted from the collective responses of neural populations to high dimensional, and 
highly structured stimuli. 
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Appendices 


A Derivation of Eqs. (3.3) and (3.4) 


Here we present the details of some of the calculations leading to the results presented in the 
main text. The computations rely on the assumption of Gaussianity. We will therefore make 
repeated use of the fact that the density A/"(x; p, £) of the normal distribution with mean p, 
and covariance £ is 


A ; 'ix: n, E l = 


\!W. 


i AH 


exp 


“(x 
2 v 




Ah 


(A.l) 


where £ denotes the determinant of the matrix £. 
'N 


Let M = 


n 


denote the cardinality of the set jC of all possible sets L. We compute 


p(x|T = 1) in Eq. (3.1) by marginalizing over s, 


p(x|T = 1) = / p(x|s)p(s|T = l)ds. 


We note that 


pWT = 1) = 5>(s|T = 1 , L)p(L) = — 5>(sir = 1, L) 


Lee 


Lee 
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Therefore, 


p( x |T = 1) = — 


1 

M 

1 

M 

1 

M 

1 

M 


I p(x|s)Jj»(s|T = 1, L)ds 

J LeC 

Y j p(x|s)p(s|T = l,L)ds 
Le£ ^ 

lim E/ •A/"(x; s, E x )jV(s; Oat, S^ L )ds 


y^A/~(x; Oat, Cl), 


Le£ 


Similarly, 


p(x|T = 0 ) = J p(x|s)p(s|T = 0 )ds 

= j Af(x; s, E x )A/'(s; 0^, £ s )ds 
= Af(x; 0jv,C), 


where Cl = £ x + Eg L and C = E x + E s . We therefore obtain 


^TD (x) = log 


p(x|T = 1 ) 

p(x|T = 0 ) 


_ / 1 y~vW(x; 0^, Cl 


= l0g 1 M 


|C| 

let 


,E 

1 Lee 


M fy Af(x; Oat, C) 


exp(--x T (C i 1 -C x )x 


(A. 2 ) 


We note that the determinant of Cl does not depend on the set L since all matrices Cl can 
be obtained from each other by permuting appropriate rows and columns. 

In the case variances and covariances are equal, we can invert Cl and C. In general, 
matrix Cl has the following form, 


(Cl)jj = 


Px^xi 

ps°s + Px.crl , if i ± j, and i,j (£ L, 


if i = j e L, 
if i = j (£ L, 
if i ^ j, and i or j e L, 


(A.3) 


We will use the Sherman-Morrison-Woodbury formula (Meyer, 2000), 

(A + UEV)~ 1 = A " 1 - A~ 1 U (E- 1 + VA-'U)- 1 CA " 1 

to obtain the inverses of Cl and C. To do so we first rewrite Cl as A L + UlEVl : When 
L = {1,2, (targets placed at the first n out of N possible locations), we have 

~( a x ~ b)I n " n r, , lT 

n (^ a 

U(7V—n)xn 

To o l 


A, = 


0 


nx ( N—n ) 


&S + y~ a)I(N-n), 


Ur. = 


V L — 


1 0 


(. NxN ) 

and 


J (Nx 2) 


E = 


(2xN) 


1 0 
0 1 


(2x2) 
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where a = p s a 2 + p x o 2 , and b = p x a 2 . 
Using Eq. (A.3) we obtain 

( 


where 


(C 




VL 


V — 


= < 


bvjgL 
7 L '■ 
V 2 .fL 
7 L ’ 
bv vj, 

\ 7 L ’ 


_ bvlg L 

1L ' 

v 2 fL 
7 L ’ 


if i = J g L, 

if i = j£ L, 

if i y j, and i,j G L, 

if i y j, and i,j p L, 

if iy j,i E L, j (£ L or i y j,i <£ L,j G L, 


v = 


V L = 


/l = a + np s a 2 s p x a 2 x v L 


Vsi 1 - Ps) + - p x y ^ a 2 x {\-p x y 

g L = 1 + p s a 2 (N - n)v, and = l + a(N - n)v + np x a 2 x v L g L . (A.4) 

Using the Matrix Determinant Lemma (|Harville 1998|), we can obtain the determinant of Cl 


det(C L ) = 


7 L 


y n v N-n 


Similarly, we compute the inverse and determinant of matrix C, 


where 


and 



v — pv 2 

-Pv 2 ■ • • 

-Pv 2 

C” 1 = 

-Pv 2 

v - Pv 2 ■■ • 

-Pv 2 


_ -pv 2 

-pv 2 ■ • • 

V — Pv 


p 

a 

1 + aNv’ 



7 un>\ 1 + aNv 
det ( C ) = „n • 


The prefactor in Eq. (A.2), is therefore 


'1 + aNv fVL\ n 
v 


7 L 


and we can compute 

x 7 C _1 x = {v — f3v 2 ) x 2 — (5v 2 x,Xj, 


N 


N 


1=1 


* 74 ? 


7 L 


ieL 


7 L 


(A.5) 


xT Cl 1x = ( Vl - bv 2 L gi\ Y x i + fv - ^ 2 /l) Y x i - - -bv 2 L g L Y x t 


Xj 


i<£L 


7 k 


i,j£L 


7 L 


bvv L Y^ XiXj 


i£L 

j£L 


7 k 


2 /l Y 


XiXj • 


hH L 
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A slight rearrangement of the terms therefore shows that when variances and covariances are 


equal, Eq. (A.2) is equivalent to 




E 

Lee 


exp 


(< A 2 (! -Ps)v l v^2 

\ i£L 


X , 


Pv 


2 vibg L 


y ^ XiXj + 2 (/3v 2 - 'Y' Xi Xj + ( pv 2 

W V 7L ; i6 frJ L 


Ilv 2 

1L 


E 


X{Xj 



B Asymptotic analysis of Eq. ( 3 . 4 ) 


Here we present some asymptotic results for the decision variable g?td( x ) given in Eq. (3.4). 
The main results are obtained for small measurement noise, a 2 . Equivalent results can be 
obtained for large external variability, a 2 . 


B.l Small measurement noise and idental distractors, p s = 1 


We first concentrate on the case n — 1. The exponential terms in Eq. (3.4) simplify to the 
following: 


~ Ps)v L v = 0, 


p v 2 _ awMst 


+ Px° 2 x 


1L Px)[Na 2 s + <j%(l + (N ~l)p x )\ 

Px[<x 2 (N - 1) + a 2 (l - p x )] 


- P*)Ws( N - !) + ~ Px)(l + i N ~ ^Px)} 

1 - Np x 


N<rl{l-p x 


+ 0 ( 1 ), 


Pv 2 - 


Vs + V x Px 


Px 


Pv 


PxCtIvlv __ 

7 l vl(±-Px)[Na 2 s + a 2 x (l + (N-l)p x )\ a 2 (N - 1) + a 2 x (l - p x )(l + (N - l)p x ) 

= iV<T|(l - P X ) + 0(1) ’ 

2 v2 fL _ Pi + a x Px 


7 l Px)[Na 2 s + a 2 x (l + (N -l)p x )] 

°s + Px(?l(l - px) 


^xi 1 - Px)[o 2 s(N - 1) + a 2 ( 1 - px)(l + (N - l)p x )] 

1 0 ( 1 ). 


N(N - 1)^(1 - p,) 

And the leading determinant term becomes: 


1 + aNv vl 


7 L 


(1 - p x )[Na 2 + a 2 (1 + (N — 1 )p x )} 
a 2 (N - 1) + cj2(1 - Px )( 1 + (N — 1 )p x ) 


N( 1 - Px ) 
N - 1 


+ ^ i?l) 
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Therefore, the decision variable becomes approximately, 


log (v\ 

/JV(1 - p„) 

1 N -1 

N 1 

-1 


ex P ) 

i= i 




2N^(l - ft) 

After rearranging terms, this can be rewritten as 


(1 - Np x )x 2 + 2 Xi ^ 


Xj 


j¥=i 


N - 1 



d TD (x) SJlog I 


N 

ex p 


i =1 


N 


Pz 


2No*(l- Pl 


1 (Xi - Xi) 2 - i ( NX 2 — (Af — l)x?) 


(B.l) 


where x is the sample mean of all measurements, and where x? is the sample mean of the 
measurements excluding the putative target, i. In the limiting cases p x = 0 and (1 — p x ) -C 1, 
we obtain the exponents given in Eq. (3.6) and Eq. (3.7) discussed in the text. 


B.2 Perfect performance when p x = 1 

We show that when p s — 1, an ideal observer performs perfectly in the limit of identical 
measurement noise. For a fixed number of stimuli, N, on “target absent” trials, ay — (N — 
1 = 0(a/c), an d hence a(x, N, p x , a x ) = 0(1). On the other hand the prefactor, 

P(N,p x ) = 0(i/e), and hence d T D (x) —>■ — oo, as e —> 0, i.e. as p x —>■ 1. On “target 
present” trials, when stimulus i is the target, then X{ — (N — 1 )~ 1 Yhj^i x j = 0(1), and 
Q!t(x, N, p x ,a x ) = 0(1/e). In this case the prefactor is still P(N,p x ) = 0(y / e), and the 
exponential term dominates. A similar argument works for the summands for which i is not 
a target. 


B.3 Single target with increasing number of distractors 

We still work under the assumption that measurement noise is relatively weak, so that we 
can use Eq. (B.l). Note that on “target absent” trials, x = s + 0(cr^/x/iV), where s is the 
true value of the (identical) distractors. We also have x* = s + 0(a%/\jN — 1). Hence, the 
first term in the exponential of Eq. (B.l) is 0(1), while the second term is 0(y/N). A similar 
argument holds in “taget present” trials. 

On target absent trials 


2a2 


( Nx 2 — (N — l)xj) = exp 


2 cp? 


s 2 + 0(VNa x ) 


for all i, 


to leading order in N. We abuse notation slightly and only use order notation on the terms 
that include measurement noise. As stimuli become more dissimilar to the target, i.e. as s 2 
increases, cy decreases exponentially, c?td(x) becomes more negative, and it is hence easier to 
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infer that a target is absent. However, the 0(y/Na x ) terms can be both positive and negative. 
Thus performance decreases with the number of stimuli. Similarly we can see that when a 
target is present 


«i(x, N, a 2 ) = exp 
aii(-x,N,p x ) = exp 


s 2 + 0(VNcr x 


1 N- 1 
2cr2 N 


1 1 + N-N 2 
'2^2 N-N 2 


when i is the target, 


(B.2) 


s 2 + 0(VNi 


Ctr 


when i is not a target, (B.3) 


again to leading order in N. As measurement noise decreases, or s 2 increases, the first term 


given in Eq. (B.2) diverges exponentially, and the terms given in Eq. (B.3) approach 0 expo¬ 


nentially. As a result c(td( x ) increases. However, the 0(\ /r Na x ) noise term increases with N, 
and an increase in the number of stimuli again decreases performance. 

If (1 — p x ) <C 1/-/V, i.e. measurement noise is strongly correlated, the first term in the 


exponential of Eq. rtB.l) dominates. Thus when correlations increase faster than the inverse 


of the number of distractors, performance increases with the number of distractors. 


B.4 Weak external structure, p s < 1, arbitrary number of targets 

We approximate each term in the exponential of Eq. 0 assuming a 2 -C 1: 


<a 2 (i ~ Ps)v L V = 


- Ps) 


- Px)[p 2 s{^ - Ps) + <X 2 (1 - p x )\ 

1 0 ( 1 ). 


-Px) 


Pv 


2 Px<7 2 x v 2 L g L 


PsV 2 s + PxO'l 


7 l [^ 2 (1 - ps) + cr 2 (l - Px))[a - 2 (1 + (A r - 1 )ps) + cr 2 x { 1 + (N - 1 )p x )) 

_ Px[o 2 8 Q + {N-n- 1 )ps + cr 2 (l - p x )\ _ 

- Px)[cr 2 s (l + (n- l)p x )(l + (N — 7i— 1 )p s ) + a 2 (l - p x )( 1 + (N - l)p x ))] 


Px 


-P*)(l + ( n ~ l )px) 


+ 0 ( 1 ), 


p y 2 _ PxO x V L V 


PsCT 2 + p x a 2 x 


1L W 2 S (1 ~ Ps) + V 2 x ( l ~ Px))W 2 (l + (N ~ l)p a ) + CJ2(1 + (N - l)p x )] 

px 

[a 2 s (l + (N — n — l)p s )(l + (n - 1 )p x ) + a 2 (l - p x )( 1 + (N - 1 )p x )( 

= 0 ( 1 ), 


and 


o 2 _ v*fL _ PsQ 2 s + PxQ 2 x _ 

1L Ps) + ^-Px)][cy 2 (l + (N ^l) Ps ) + al(l + (N-l)p x )\ 

_ PbQ 2 s (1 + (n - 1 )p x ) + Px(T 2 x (l - Px) _ 

[Vsi 1 - Ps) + cr 2 x ( 1 - Px)]W 2 (l + (N — n — l)p fl )(l + (n - 1 )p x ) + o*{\ - p x )( 1 + (N - l)p x )] 
= 0(1) 
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We also approximate the leading coefficient of the exponential term in Eq. (3.4) as: 


'1 + (ps<?s + Px°l)Nv fVL\ n 

V 


7 L 


_(1 - Px)W 2 s (l + {N- l)p a ) + a 2 ( 1 + (N - l)p x )]_ 

[a s 2 (l + (N -n- l)p s )(l + {n- l)p x ) + o-2(i - Px )(l + (N ~ l)p a: )] 


~ Ps) + <^(! ~ Px) 

^(! - P*) 


(1 - p x )( 1 + (IV - l)ps) f Ps( 1 - p B ) 
(1 + (IV - n - l)p s )(l + [n - 1 )p x ) V^*(! - Px) 


+ 0 ( 1 ). 


Combining above terms, Eq. ( 3.4[ ) reduces to the following expression under the assumption 
of a 2 x < cr 2 ,p s < 1, 


d TD (x) - log — 


(1 — Px)( 1 + (N — 1 )p s ) 


^ 2 (! -Ps) 


E 


x > exp 
Lee 


M\j (1 + (N - n - l)p s )(l + (n - 1 )p s ) - Px) 

1 I X ■> o px 


2a 2 x (l - p x ) 


.ieL 


E 


1 + (" - !)«" Ei 


OC J 


Special case: n = 1 In case of a single target, set L has only one element and C = 


{1, 2, • • • , N}. In this case, Eq. (3.9) reduces to a much simpler expression 

/ ^U-PsJU + CV ~l)Ps) ' N 
N* 


1 t N, , I 1 (1 — Ps)(l + (N ~~ l)Ps) 

d TD (x) - log ( -\l , -2^ ex P 


al(l + (N-2)p s ) 


i =1 


XJ 


2u2 


This expression is independent of p x . Hence, the decision boundary, and the performance of 
an ideal observer is unaffected by measurement correlations. 


B.5 Near perfect performance with n > 1, and p x ~ 1 


We first assume that T — 1. From Eq. (3.10), if Lt is the set of targets, then this expression 


is approximately zero, and the exponential is approximately unity. When L is a set not 


consisting of all targets, then the expression in Eq. (3.10) has expectation greater than zero. 
Then 


E 

Lee 


n 


exp 


2o-2(l - p x ) n 


E 

ieL 



!+ ex p|‘ 


na\ 


LeC\L T 


2(j2(l - p x ) 


->■ 1 


(B.4) 


The prefactor in Eq. (3.9) diverges since the exponential dominates. If T = 0, then Eq. (3.10) 


will be greater than zero for all sets, L. Thus o?td(x) — > —oo with p x —y 1. 
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B.6 Asymptotics for large N 

Here we develop asymptotic results for the decision variable when the number of stimuli, N 
is large. We assume that there a fraction K of the stimuli are targets, so that there are 
NK targets and (1 — K)N distractors. To simplify notation we write c x = a 2 (l — p x ) and 
c s = a 2 s {l — p s ). We find that 

c. 


(1 - p s )v L V = 


V ‘ 1 P ~ Tl (^) P ’ alSL 1 = 

2 fa 1 TZj i i 

V P - Px(T x 

V 7 L V 


C x (Cs ~t~ Cx) 

11 11 


N c s + c x NK c x 

1 1 o( 1 

Nc s + c x + [ N* } 


+ ° { m ) 


v 2 [P - fh = U,- 
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1 1 \ 1 r( 1 
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Cx 


KN 


1 {px - 1 )(psV 2 s + PxP 2 x ) 

N (K - l)Kp s p x a 2 s 


We can therefore group the terms in the exponential of the decision variable given by Eq. (3.4) 
as 


Qe(Cs T Cx) 


I>* 2 + 
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A simple reorganization of the terms yields Eq. (3.11). 


C Mean stimulus orientation - left or right discrimina¬ 
tion task 


An observer is presented with N stimuli on every trial. For concreteness, we can think of the 
stimuli as bars, with orientations s = (si, S 2 , ■ ■ ■ , sjv)- The task is to decide whether the mean 
orientation of the set is to the left (a condition we denote by C = —1) or right (C = 1) of 
the vertical. The observer makes a decision based on the measurements, x = (aq, x 2 , ■ ■ ■ , x^). 
Stimulus orientations are drawn from a multivariate normal distribution with mean vector, 
Otv, and covariance matrix, E s , 

p(s)=AT(O iV ,E s ), (C.l) 


with E s defined in Eq. (2.2). As in the target detection task, we assume the measurements 
follow a multivariate normal distribution with mean vector and covariance matrix specified in 
Eq. 0. 

An ideal observer performs the task by making a decision based on the log posterior ratio, 


dMOD(x) = log 
= log 


p(C = l|x) p(s > 0|x) 

p(C = — l|x) °®p(s<0|x) 
p(x|s > 0) p(s > 0) 

p(x|s < 0) p(s < 0) ’ 


(C.2) 
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N 


where, s = s* denotes the mean stimulus orientation on a trial. If ^modM > 0, the 


i= 1 


observer infers C — 1, that is, the mean stimulus orientation is to the right of the vertical. 


We compute Eq. (C.2) by marginalizing over s and applying Bayes’ rule, 


p(x|s > 0) = J p(x|s)p(s|s > 0)ds 

= j p(x|s)p(s > 0|s) 

1 


p(s) 


p(s > 0) J - s> o 

1 f 


p(s > 0) 
p(x|s)p(s)ds 


ds 


A/"(x; s, S x )A/’(s; 0 N , S s )ds 


p{s > 0 ) J- s> o' 

= yiTo) L m ( s; (I + E * E -^ Xl (E;1 + s - Tl ) *■ 

where z c is a normalization constant. Similarly, we compute p(x|s < 0) and obtain 


4iod(x) = log 


-1 i v — —1 


ds 


/»>„ V (s; (I + W) x. (S; 1 + S 

/,<„ v (s; (i + EkE-t 1 x, (sy + s-i)- 1 ) * 


(C.3) 


By symmetry, it is easy to see that the decision boundary in the space of measurements is 

N 

given by the hyperplane x — 0, where x = is the sample mean of the measurement. 

i= 1 

An ideal observer therefore bases the decision only on the sample mean. Therefore, we have 
x > 0 =>■ s > 0. 

In order to understand the negative impact of noise correlations on the performance of an 
optimal observer on this task, consider x = s + £ where £ = (£i,£ 2 , • • • ,£tv) ~ W(0, £ x ), so 
that 

+ =S + £- 


In this case, the variance of the mean noise scales with increasing noise correlation strength, 


Px • 


Var(£) = 


~ Px) 

N 


+ Px& x 


As the variance increases, the overlap between the conditional distributions p(x\s > 0) and 
p{x\s < 0) increases. It is therefore more difficult to tell which condition the measurement 
comes from, and performance deteriorates. 
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